Integrating recognition confidence scoring with language understanding and dialogue modeling

In this paper we present a method for integrating confidence scores into the understanding and dialogue components of a speech understanding system. The understanding component of our system receives an n-best list of recognition hypotheses augmented with word-level confidence scores. The confidence scores are used by the understanding component to hypothesize when words in a recognizer’s n-best list have been misrecognized. The understanding component has the ability to predict the semantic class of misrecognized words based on the surrounding context and also to suggest when key words which may have been misunderstood should be re-confirmed by the user. The output of the understanding component is passed onto a dialogue control component which can act on various suggestions made by the understanding component. To evaluate the system, experiments were conducted using the JUPITER weather information system. Evaluation was performed at the understanding level using key-value pair concept error rate as the evaluation metric. When word confidence scores were integrated into the understanding component, the concept error rate was reduced by 35%.